GFX Sensations 1/Graphic Sensations - Volume 1.iso/tools/amiga/editors/vtutil1.lha/Imploder_Doc

Text File | 1993-12-29 | 73.2 KB | 1,796 lines

Peter van Campen: I had to put the manuals of this program in one
file to save diskspace.

***************************************************************************

This archive contains the Imploder 4.0 distribution.

-All files directly related to the Imploder have been moved into the
 "Imploder" directory and subdirectories.
 The Imploder allows you to reduce the size of executable files while having
 them retain their full functionality.  There are other "crunchers" or
 "packers" available for the Amiga, but none are as mindful of the
 complexities of your Amiga system as the Imploder (IMHO). In addition to
 this, its algorithms are more efficient, both in terms of speed, and size
 reduction.

-The FInf drawer contains a versatile directory lister and file type
 recognizer. This one is bursting with options, many of which are
 sorely needed but not present in any of the other listing utilities
 I know of. Of course it recognizes whether files are Imploded or not.

-The DImp drawer contains a disk-archiver that makes use of the Implosion
 algorithm. Special features include the creation of self extracting and
 explaining disk archives, support for any device with a floppy like track
 layout, and the flexible selection of cylinder ranges.

***************************************************************************

How to operate the Imploder should be fairly obvious to the average Amiga
user. So I'll try to highlight only those aspects of the user interface
that aren't immediately obvious, and of course this document will explain
how certain selections influence the processing behaviour.


------------------
The User Interface
------------------

** Configuring the Imploder **

When you run the Imploder, either from the CLI or WorkBench, you'll notice
the music and NTSC size screen. This setup is configurable by way of
commandline switches or tooltypes. Note that, under Kick 1.3 or less, when
one sets a tooltype using the WorkBench's info option, the appended "=" is
required for proper recognition.
The default setting is enabled music without modifications to the filter or
NTSC/PAL mode. However, you can disable the music using the "NOMUSIC"
commandline switch, or the "NOMUSIC=" tooltype. The lowpass filter can be
disabled for less muffled sound by specifying "NOFILTER/NOFILTER=".
To people with PAL Amigas, the shrunken NTSC screen might look ugly, however
if there is a fat Agnus chip in your machine, it can be toggled from a 50Hz
to 60Hz display refresh rate. Setting "NTSC/NTSC=" will cause the Imploder to
do this whenever its screen gets upfront.

In addition to this, there's the SHORTROOT configuration switch. This
option is related to using the library. See the "library" document for
further information on this.


** The File Requester **

Whenever there exists the need to specify a filename, a directory window will
appear in front of the message window. Only a few of its options need
clarification:
-Switching on the ID toggle gadget causes the directory window to display
 the type ID's of all files. This is useful, for example, to check if files
 have already been imploded. This option will increase the time needed to
 access a directory though.
-Clicking on the proportional scroll gadget on the left will cause the list
 of filenames to be sorted.
-Switching off the .INFO toggle gadget causes the directory window not to
 display the *.info files used by the WorkBench's icon system.
-Selecting a file is done by placing the mouse pointer on top of its name and
 clicking the left mouse button. In addition to the normal ways of confirming
 your selection, you can double-click on a filename.
-Next to the immediately obvious ways of scrolling through a long directory,
 one can place the pointer within the filename window, press the left button,
 and while holding it down move the mouse above or below the filename window,
 try it; it works great once you get used to it.
-One does not have to wait for the entire directory to load before selecting
 a file or entering a sub directory.


** Keyboard Equivalents **

Many functions selectable by mouse also react to keys. The menu options have
the usual right-amiga+key shortcuts. In addition to this;
- Hitting any key removes the about requester displayed during startup.
- Use the escape key to exit the Imploder.
- The "S" key starts processing.
- When the merge/compression parameter window is present, you can select
  the compression mode using numeric keys, and increase/decrease the merge
  threshold using the "+" and "-" keys. You can cancel using ESC and proceed
  by hitting RETURN.
- The deplode query requester reacts to the "Y" and "N" keys.


** Loading / Saving **

When you have specified a file for the Imploder to load, processing will
commence (detailed below), and once finished the file requester pops up
to query you for the destination.

The source and destination paths are maintained in a special way that
minimizes the chance of the destination not being what one wants. You
could e.g. implode files from one directory and store them in another,
or overwrite them in the source directory.
In the first case it would be annoying if the destination directory defaults
back to the source; you'd have to change it every time you want to write an
imploded file. In the latter case, it'd be annoying to change to a different
source directory and find out that the destination still points somewhere
else.

So the logic the Imploder uses is as follows; if you change to a different
source directory, the destination directory will follow the source directory
only if the destination is equal to the source.


** Batch Mode **

The Imploder can compress a series of executables by allowing you to specify
multiple files by way of ?,#,* wildcards. All non-imploded executables
matching the pattern will be processed. After entering the wildcards, the
"merge and compression" parameter window will appear. The options present
there will be explained below. The important thing to note here is that
these selections will be global to all files processed during batch mode.
Next, you'll be asked to specify a destination directory. This is were all
batched executables will be saved after implosion.


----------
Processing
----------

Now you're familiar with the user interface, let me get on with explaining
how the Imploder processes programs. The implosion processing sequence
consists of four steps;
-Loading & Format Verification.
-Hunk Merging & Reloc Table Cleanup.
-Implosion.
-Decompression code installation, Overlay table adjustment and Saving.


** The Loader **

The loader reads the selected file. During this process, the executable
file is analyzed. Format defects cause the loader to either report an
error, and abort processing, or to give a warning while ignoring or
repairing the defect.

After having loaded a file, the "merge and compression" parameter window
will appear. You can click the topright icon to collapse this window and
review the status information produced by the loader.


** The Merger **

The merger is disabled by default. Most executables produced by modern
compilers/linkers don't require this option, so if you're impatient
you can skip to the next section.

If selected, the merger will try to coalesce the hunks in a program to hunks
of upto the merge threshold in size. If used in a constrained manner, this
will reduce the size of the program, and the chance of memory fragmentation.
If you specify a large merge threshold however, the resulting file will need
large contiguous regions of memory in order to load, thus reducing the chance
of it being able to be loaded into a fragmented system.

Merging an executable file might break it. This is because certain programs
make assumptions about the format of their segment list. The most notable
members of this group are BCPL programs and a few libraries and devices.
Hunk merging is therefore automatically disallowed for these. Still, some
other programs, especially selfdetaching programs might dislike being merged.

Regardless of whether a program has been merged or not, a reloc table
cleanup routine is executed upon the file. This deletes empty reloc tables,
coalesces matching reloc tables, and finally sorts the relocation offsets.
This is of no consequence to how the program will look after it has been
decompressed and relocated.


** Compression **

The compression algorithm can operate either in turbo mode or in normal mode.
The normal mode requires no additional memory, whereas the turbo mode needs
some 300K of additional hashing buffers. The turbo mode kicks-in automatically
whenever the required amount of additional memory can be allocated. It is
about ten to twenty times faster than the normal implosion mode.
Note that the parameter window will report whether or not the current memory
configuration allows the turbo to be enabled or not. You might try to free a
few resources in order to make the target. Whenever you select a compression
mode (gadgets 0-8), the turbo capability will be reevaluated.

Various compression modes can be specified. These modes vary from zero
(range  = 128 bytes) to eight (range = 18K). The range related to each
compression mode determines the maximum distance searched while looking for
redundant data. The higher compression modes therefore have a better chance of
compressing data.

The time required to compress a program increases proportionally to the
maximum distance in case of non-turbo operation. The processing speed during
turbo operation however, is only slightly affected by varying the compression
mode.

So it only makes sense to fiddle with the compression mode if you don't have
enough memory to have the Imploder run in turbo mode, and therefore want
to speed things up (at the expense of a the compression efficiency).

For all other instances it suffices to leave the compression mode at its
maximum value (eight); for small executables the mode will automatically
be scaled down to what the Imploder thinks is probably the most efficient
one for the file at hand.

During compression, the level meters to the right will display various
relevant parameters. Going from left to right these are:

1. Percentage of program data still to be compressed.
2. New size of executable in % of the former size. Reevaluated continuously.
3. Skip. If a large chunk of non-compressable dat is encountered within
   a program, its level will start to rise. When it hits the top, the
   encoding limit has been exceeded. This is a rare occurance.
4. Length. Gives a measure of the redundancy of the data currently being
   processed. High levels indicate good compression.
5. Distance. Gives a measure of the non-locality of current redundancy.


** Decompression Code Installation **

All imploded executables require some additional memory in order to be
able to decompress. The additional amount of memory required is about 50%
of the original program size plus a constant amount of buffer space depending
on the compression mode, and never larger than 20K.
After decompression this memory is freed without causing memory fragmentation.
Furthermore, the normal distribution of program-data across hunks is retained.
This allows for the loading of imploded files into fragmented memory, and
the proper distribution of hunks across chip and fast memory.

The Imploder is able to install 3 different decompression algorithms into
imploded executables;
Library, Normal and Overlayed.


LIBRARY IMPLODED files are the default, they are generated when;
-The "Compress" gadget is enabled.
-The "Library" gadget is enabled.
-The processed executable isn't overlayed.

To be able to use library Imploded files, copy the explode.library to your
LIBS: directory. When you run a library imploded program, the decompression
code in this library will be called. The library code is fast and removes
the need of appending decompression code to every imploded file.

The library has several special useful properties discussed in the "Library"
document, but for simple use it suffices to make sure the library is in LIBS:.

Pure programs may be library imploded.


NORMAL IMPLODED files are generated when;
-The "Compress" gadget is enabled.
-The "Library" gadget is disabled (this is NOT the default).
-The program being processed isn't overlayed.

Normal imploded files have decompression code appended to them. A normal
imploded program will run just like the original, and is independent; in
contrast with library imploded files, normal imploded files don't require
libraries or other special files to be present in your system.

Disadvantages of normal Imploded files are that they are a couple of hundred
bytes larger than library Imploded files, and their decompression speed is
significantly slower. This is due to need for space optimizations in the
decompression code.

Once you normal-implode a pure program it's no longer pure.


OVERLAYED IMPLODED files are generated when;
-The "Compress" gadget is enabled.
-the processed executable is overlayed.

Overlayed programs are executables with additional appended hunks that are
loaded during runtime. The Imploder will automatically recognize whether a
program is overlayed.

Overlayed files are rather loosely defined; basically the program is passed
a bit of information that allows it to get at the appended hunks, but the
means of doing so is left up to the programmer, though there is a standard
technique specified by CBM which is used by the majority of overlayed
executables.

Because the Imploder decreases the size of the file, the offset of the
overlay data also changes, and this must be corrected for. The Imploder
knows how to do this for the standard overlay format. You'll be notified
when the format isn't as the Imploder expects.

Thus imploding overlayed files is not something which is guaranteed to
succeed, so take heed; you should only implode overlayed files when you
know what you are doing, and are willing to verify the results.


---------
Deplosion
---------

If you select an already imploded executable for processing (the ID gadget
in the directory window allows you to see if files have already been imploded,
at the cost of slower listing), you will be asked if you want to deplode it.
If you confirm this query, the executable will be restored to a normal
non-imploded format. Note that this format isn't necessarily bitwise
identical to the original; the reloc tables are sorted, and you might
have applied hunk merging (which is irreversible). Also, any symbol or
debug hunks present in the original will have been stripped.
Still, if you run the program, things will look the same to it, and
this is ofcourse what counts.

Note that there is a CLI based "Deplode" command available in the Tools
directory which does basically the same thing. This is purely a matter of
convenience ment for people that think of the CLI as convenient.

***************************************************************************

The Imploder was conceived, written and performed by:

Albert-Jan Brouwer    -    Programming, documentation.
Peter Struijk        -    More programming, less documentation.
Paul van der Valk    -    Music-programming and composition.
Erwin Zwart        -    Graphics, aesthetic lay-out.


Please direct queries, comments, bug reports, and other things that can
not be resolved by rtfming to:

UUCP hp4nl.nluug.nl!cbmnlux!ecl001!ajbrouw
UUCP cbmvax.commodore.com!cbmehq!cbmnlux!ecl001!ajbrouw
FIDO: 2:281/614 (up for freq sometime fall '91)


Legal mush:

The Imploder is Freely-Distributable, as opposed to Public Domain.
Permission is given to freely distribute this program provided you
include this documentation and other related files, and no fee is
charged in excess of reasonable media and mailing costs.
All programs in this distribution have been enforced and mungwalled.


P.S.

We've been promising a 4.0 version for over a year now, well here it
is. Things got delayed slightly. We apologize for the inconvenience.

***************************************************************************



Changes V3.0 -> V4.0
--------------------


Major:

- Finally a complete and up-to-date well documented release with all
  of the support utilities.

- Basic support for batchmode processing.

- Radical new music composition by Paul van der Valk.

- Erwin Zwart has completely redesigned the graphics to conform to the
  2.0 look. Aesthetics and functionality have improved significantly.

- Defaults to support for a "safe library root". This is larger startup code
  for library imploded files that warns when no library is present. You can
  override this setting.

- The "Protect" option has been discontinued. See the "philosophy" document
  for the rationale on this.

- Mouse hater support by way of a CLI based Deplode command, and keyboard
  equivalents throughout.



Minor:

- Support for "pure-imploded" files removed; If you know how to make a
  program resident, you know how to use the library.

- The "Library" gadget now defaults to on.

- The "Merge" gadget now defaults to off. Improved compilers/linkers
  plus the much more frequent appearance of selfdetaching programs have
  made this option less important and more dangerous respectively.

- Added tooltypes and commandline switches for specifying the setup
  options.

- The copperlist has been tuned down a bit, and the screen is now
  dragable.

- The explode.library has been enhanced/sped up a bit.

- Removed bug that caused Intuition to read from location zero. This would
  occasionally crash the machine when location zero was nonzero.
  This bug was already removed in V3.1 (thanks to Arnout).

- All stuff has been enforced and mungwalled so now we've even better
  confirmation that no serious bugs are present.

- Improved overlay file recognition. No longer expects overlayed files to
  have additional space in the hunk table. Searches for the magic longword.

- Decodes ARP resident program tags. Warns against stack gobblers.

- Data hunk ID now preserved upon deplosion. Inconsequential, yet neater.
  Very stringent format checks in the old deplosion routine barf on the
  extra bit required for this, but the old explode library doesn't mind,
  so in that sense it is backwards compatible.

- Lots of other small changes.

***************************************************************************


    Introduction
    ============

The Imploder allows you to reduce the size of executable files while having
them retain their full functionality.  There are other "crunchers" or
"packers" available for the Amiga, but none are as mindful of the
complexities of your Amiga system as the Imploder (IMHO). In addition to
this, its algorithms are more efficient, both in terms of speed, and size
reduction.

Basic operation is clicking and using pull-down menus. Though operating
the Imploder isn't hard, the complexities of the Amiga give rise to the
need for imploding different types of executable files in a number of
different ways. Please note that not reading the documentation won't
inhibit you from using the Imploder; it automatically detects, and reacts
to almost all exceptions, and for the vast majority of files it suffices
to simply hit the "Proceed" gadget after loading has finished.

Still, the Imploder does have a few - quite useful - options that require
you to have some understanding of the issues involved. Notably, it would
be preferable for you to read the documentation before making use of the
explode library or the merge option.

----

Judging from some of the reactions we've received, the old documentation
didn't do a good job in communicating essential information to the user,
so it has been revised and split into several parts for easy reference.

Here's a list:

- Authors;
  About the authors, legal status, distribution etc.

- Changes;
  Highlights, newly implemented features and future intentions.

- Manual;
  How to operate the Imploder.

- Library;
  Information on the explode.library and related issues.

- Techno;
  Some in-depth information on the Imploder's internals for those tech
  types that want to know it all.

- Philosophy;
  This discusses the Imploder's design choices and intended use.

***************************************************************************


** Features **

Library-imploding files is preferable, not only because the result is
smaller, but also because the library is faster, and able to patch the
AmigaDOS executable file loader (LoadSeg), thus decompressing ANY type
of load file as soon as it is loaded into memory, e.g. fonts, devices,
libraries, etc.

If you only library-implode program files, the library will be loaded
when you run a library imploded program for the first time. However if
you wish to implode non-program executables, you'll have to make sure
the library is already resident because these won't load properly until
LoadSeg has been wedged. If you're not yet using the 2.0 OS, there is a
slight complication explained in the section at the end of this document.

To make sure the library is resident, we've provided a small program that
can be run at the head of your startup sequence. It's called "ExpLoad"
and resides in the tools subdirectory. All it does is open the explode
library. Once this is done, the library hooks itself into DOS and remains
resident, even when a system flush occurs.

Note that it suffices to run a library imploded program to make the
library resident, but this is less visible so you might forget about it
when you modify your startup-sequence later on.

Once you've made sure the library is resident, you may merrily go around
your system and library-implode libraries, devices, fonts and even keymaps!

By default the Imploder appends a foolproof piece of startup code to
library imploded files. This checks whether the library is present.
If not it will print an error. If started from the WorkBench a tiny
window will be opened to display the error message.

However this error checking takes up a bit of space, and is redundant
when you make sure the library is resident. For this reason you can
specify the SHORTROOT tooltype/switch. When set, the Imploder will
append only short startup code to library imploded files. This saves
about 250 bytes for every executable.

If library imploded programs with a short root do not find the explode
library they will keep on trying to open it until they succeed. These
programs will therefore hang until the LIBS: directory contains the
library.

Still, using the SHORTROOT switch has its advantages. The reason it is not
enabled by default is to make sure that people have read this document
first.


** Pre-2.0 problems **

If you are NOT using 2.0 or some newer OS, there are a few additional
problems related to BCPL braindeadness, so if you're still running 1.3 or
less, READ THIS!

Under a pre 2.0 OS, if you run a library imploded program generated with
the SHORTROOT option from the WorkBench, while the library hasn't been made
resident yet, the system will guru. This is a problem caused by the OS.
The default library startup code checks for this condition, and will exit
cleanly after reporting an error.

So under 1.3 or less it is advisable to make the explode.library resident
before the WorkBench is active, e.g. early in your startup sequence, even
when you're using library imploded non-program executables;
The first invocation of a library imploded program might happen from the
WorkBench.

In addition to this it is best not to library implode handlers under pre
2.0. See the Techno document for an in depth explanation of these issues.


** Conclusion **

Library implosion limits the possibility of passing on one's executables
to other people/systems who/that do not have the explode.library installed.
I think this doesn't really matter because I strongly believe people
should be left the option of deciding in what way, if at all, they are
going to compress/install executables. (See the "Philosophy" document
for a more extensive discussion of this issue).

Evidently, using library imploded files is the most transparent to the
system. We personally recommend using mostly library implosion, and only
in special cases normal implosion, or overlay implosion for the odd
overlayed file one might encounter.

***************************************************************************


This doc may sound slightly grumpy in places, still this is how we feel about
things, so there ;-)


Intended use
------------

The Imploder is intended for creating extra space on your system disk
while maintaining full functionality. To achieve this goal, we've tried
to make the decompression process as invisible and fast as possible.

Thus the Imploder doesn't support any annoying colour flashing, and it
has a high speed of decompression. On a vanilla 68000 Amiga, "explosion"
speed is about 30-50 K/s, depending on the type of compressed code. Also,
the explode.library is a bit faster than the explosion routines appended
to stand-alone "imploded" files.

You should take this speed issue into consideration when determining which
executables to implode. For floppy users, the startup times will mostly be
FASTER. Users with fast harddrives however might want to limit themselves
to imploding only infrequently used executables. It is therefore very
useful to floppy users, yet harddrive users can also add a few megs to
their bit budget.

Evidently, the optimum use of the Imploder will vary from system to system.
A3000 owners for example won't even be able notice programs exploding.

Because of this, over the years, the emphasis of the Imploder's intended
use has moved away from Imploding being a once in a program's life time
operation, suitable e.g. when an author wants to distribute his program.
Instead we now feel that every user should be able - when installing a
program - to decide whether or not to implode it, and in what way.


The user's responsibility
-------------------------

So the issue here is freedom of choice. And it is responsibility of the
user to apply the Imploder in a sensible fashion.

To get a feel for what I'm talking about simply look at what happened
when people started distributing Power-Packed text files. Many people
didn't like using the PPMore reader or simply didn't have it. This
kind of thing can be utterly annoying, and the Imploder has the
potential for the same kind of abuse.

We therefore recommend limiting the use of the Imploder to compressing
things installed in your system. So if you must pass on a program to
someone else, use the original archive. This keeps everyone happy,
including the program's author.

Another point is that when receiving programs from PD sources, people
should be able to easily check if programs contain file viri or other
unwanted things. If an executable is imploded it'll have to be deploded
before one is able to examine the code. So don't distribute compressed
executables of _any_ type.

Still, there was one action we could take to encourage the kind of use
we recommend;
In previous versions one could select a "protect" option that prevented
a program from being deplodable. The intent of this option was to protect
an author's work before he distributed it. In reality people started using
it to protect dirty hacks or programs containing file viri.

Evidently this is incompatible with our philosophy. Thus, the support for
protecting programs has been discontinued as of version 4.0. In addition
to this, the Imploder now allows decompression of any protected files
left over from the past.

***************************************************************************

This document deals with the inner workings. If you're a tech type, and are
interested in somewhat deeper lying aspects relating to how the Imploder
operates, chances are you'll find it here.
Subjects covered are:

- Compression
- Decompression
- Reversibility
- The Library
- Overlayed Files
- Merging
- ARP
- The Music
- The Copperlist
- 68040 Cache Coherency


** Compression **

The Imploder (we only recently learned :-) does LZ77 like compression with a
per-mode static Huffman coding step on the various parts of the skip, offset
and length tuples. Due to the efficient encoding, a tuple can require less
than 12 bits, and thus strings of 2 bytes length are encodable with a decent
gain (given small Huffman patterns corresponding to likely circumstances).

To speed up the string searching, the turbo mode causes the accessible strings
to be indexed using a hashing table. However, the fact that strings with a
minimum size of two bytes are still potential candidates for compression,
requires the hashing function to necessarily be rather simplistic. When the
implosion algorithm processes highly redundant data, entries in the hashing
table tends to get very imbalanced, causing a reduction in performance.
For most types of data, notably executables, this isn't the case though.


** Decompression **

The goal of decompression is to reproduce the segment list of the
original program. This is a linked list of data/code/bss hunks, some
of which require being positioned in chip memory.

The decompression code will have to fill the data and code hunks with
the decompressed data, and, if required, perform relocation of absolute
references.

The Imploder lets LoadSeg allocate all target hunks (as BSS). These
are at the start of the hunk list. This is followed by a hunk with
the decompression code (only for non-library imploded programs),
a compressed data hunk (normally about 50% of the static data size,
depending on compression gain ofcourse), and a decompression buffer
(only upto 17K in size).
As you can see, no allocations need to be done, so the exploding
process will never fail.

During decompression time, data is decompressed from the compressed data
buffer into the decompression buffer until it has filled. It then empties
this buffer by filling hunks and processing reloc tables.
When the buffer has been emptied, the process repeats until all data has
been scatter-decompressed across the target hunks.

Now you might ask, why not directly decompress to the target hunks
instead of using a decompression buffer?
This is because the the Imploder uses the decompressed data to
decompress, and needs to be able to easily access it (for speed).
Referencing data distributed across several hunks is much more
cumbersome.

An added bonus of having separate source and target memory regions is that
a notation can be used that doesn't gain under rare circumstances, but on
average yields better results (No chance of source/target pointer
collision).

When explosion has finished, the decompression buffer, code and data
hunks are freed, and memory usage reduced to what it would have been
for a non-imploded version of the program.

People often compare the Imploder to the Power Packer. The Imploder
decompresses faster and looks cooler [:-)], but the most interesting
differences lie in the implementation of the various steps of the
decompression proccess. So let's contrast the advantages of the
Imploder's approach with the Power-Packer's implementation.

- By having LoadSeg allocate all memory as BBS hunks, explosion will
  never fail.
  The Power-Packer on the other hand allocates hunks. If it fails it
  will simply exit. Power Packed programs launched from the WorkBench
  thus won't reply the startup message, which will be left dangling in
  memory forever.

- Memory doesn't get fragmented. The explosion related hunks are at
  the end of the seglist and thus were allocated (by LoadSeg) AFTER
  the target hunks.
  This isn't true for the Power-Packer. It does leave a hole in your
  free memory list when it frees its decompression stuff.

- Additional memory usage is only about 50% of the static data size +
  the size of the decompression buffer, which is always small relative
  to large programs (maximum 17k).
  So a 30K program might require 62K to decompress (30+15+17), a 300K
  program will require 467K (300+150+17), assuming a 50% compression
  reduction.
  The memory usage report generated after a program has been imploded
  includes BSS hunks. I've discussed only static data here. BSS hunks
  don't require any extra memory usage of course.
  Power-Packed files require a buffer as large as the original program
  for both compressed data storage and decompression. Memory usage is
  therefore always about twice the static data size (again ignoring BSS)
  while for the Imploder it drops to 1.5 for executables large enough to
  matter memory wise.

(In this comparison I'm talking about executables as produced
 by Power-Packer version 3.0b.)

Non-library imploded programs have a small first hunk that calls the
decompression code hunk at the end, and frees these last three hunks.
For library imploded programs this freeing occurs in the library, so no
preceding hunk is needed.


** Reversibility **

Before compression the Imploder preprocesses executables. It kicks out all
the redundant stuff by merging subreloc tables referring to the same hunk,
sorting relocs (improves compression), removing debug symbols etc. etc.
This is what all those info blurbs in the text window are about.

So the deploded executable isn't guaranteed to be byte by byte identical
as far as loadfile control codes are concerned.

What is guaranteed is that the memory images created when the original,
imploded and deploded program versions are loaded are identical.

So the deplosion process isn't 100% reversible. Normally this is no
problem. The reason for uncrunching is mostly wanting to recompress
an executable using a different compression mode, or having a quick
peek at the code e.g. when applying a patch with something like
NewZap.
If however you expect to need the debug symbols, or (important)
require the executable to be in the _exact_ original format in order
to have things like lpatch updates to applied, you're out of luck
if you've only got the compressed executable. So always keep the
original archives/disks!

This is yet another argument for retaining the original archives.
The Imploder is an online space creator not a distribution
archiver (See the "Philosophy" text).


** The Library **

The library code has been unrolled a bit, and optimized here and there
in order to achieve optimal performance. This makes it faster than the
normal explosion algorithm.

If you library implode a program there is NO way in which the program,
after explosion, will be able to notice. If you make sure the library
is resident, this is also true for any executable file loaded for any
purpose by any program.

For normal etc. imploded programs the startup/cleanup hunk mentioned
at the end of the "decompression" section might be detected if a program
goes through contortions involving finding the segment list via murky
DOS structures instead of simply doing PC relative hunk start referencing
which also works from the WorkBench.
I haven't encountered any programs that do this. Still this is yet another
reason to use the library; there is not even the slightest chance of it
being incompatible with an executable.

Note that the Loadseg vector is patched in an "intelligent" manner; it
will install fine for pre 2.0 kickstarts (braindead jumptable format)
as well as in BCPL free systems (2.0+)

Under pre 2.0, when a library imploded file is run from the WorkBench, and
the explode.library isn't resident yet, Exec will try to load the library
from disk. The process's message port however is in use by the WorkBench
reply message, and until it has been replied, it cannot be used by the
DOS in order to send packets. Thus the DOS gurus.

Also, BCPL code doesn't jump through the the library vector. The only
structural problem with this are handlers. These are loaded by the DOS,
and the DOS is BCPL code, again ONLY under < 2.0. Under 2.0 the library
works just like intended when it was first conceived. Transparently that
is.


** Overlayed Files **

The Imploder compresses the load part of an overlayed file as if it were a
normal executable file. Subsequently, the overlay table and the overlayed
program section are appended.
It then tries to adapt the overlay table. Because different types of
overlay supervisors are in use, the format of the overlay table isn't
known to the Imploder. The only assumption made is that the overlay table
contains seek-offset longwords, at longword aligned locations, that point
into the file to the hunk_header ($3F3) identifiers of the overlayed
program sections.
This is how the standard overlay manager operates, but nothing prevents
a programmer with sufficient technical knowledge to create a novel overlay
format (e.g. selfextracting DImp files).

If the Imploder finds one of these offsets, it is adjusted by the amount
the initial load part of the executable file has compressed.
The deplosion algorithm also tries to find these offsets when restoring the
overlay table. Thus there is always a very small chance that the imploder
will adapt a longword that was never meant to be an offset.

An overlayed file gets its information from the loader in four longwords,
at the start of the first hunk. In an imploded overlayed file, this hunk is
the root hunk, and after decrunching these longwords are adjusted and moved
into the first hunk of the actual program (the second hunk of the seglist).

Evidently this process can never be 100% deterministic, so take heed and
test any overlayed programs you've Imploded. Or don't use overlay implosion
at all if you can spare the bits.


** Merging **

Though modern linkers/compilers typically produce executables with one code
hunk and one data hunk, there are still some old executables and less evolved
linkers around. The merging option was implemented when executables with
sufficient hunks to cause a lot of redundancy were still commonplace.

Every hunk requires a longword in the allocation header, plus a hunk ID,
load size, and hunk end ID. That's 16 bytes per hunk, and thus saved for
every merge action. Doesn't sound like much, but generally hunks also
have reloc tables. These waste a lot more space, especially with references
to a lot of different hunks, though there's no easy equation.

The merging step merges matching hunks (data-data, chipdata-chipdata,
code-code) into hunks of upto the merge threshold in size. The actual
size is of course determined by the sum of the sizes of the composite hunks,
and may very well be a bit less than the specified threshold.

Obviously this process discards redundant data in an irreversible fashion,
so deploding the executable won't reverse it.

Lots of tiny hunks cause memory fragmentation, but increase the chance of
the program being able to load when the system is fragmented, and low on
memory. Thus there is a kind of optimal balance that varies from system
to system. In general it can be said that hunks less than 10K or more than
100K are "bad".

Another factor is that loading a program with many tiny hunks causes the
LoadSeg function to issue double as many tiny read commands, thus bogging
down the speed with which an executable can be loaded into memory.

For simplicity's sake, I've chosen for the Imploder to process executables
within a single buffer, without the need for additional backup buffers.
Thus, removing redundant information, and copying hunk data during the
merging and reloc cleanup process involves moving or mirroring large parts
of the buffer. This is why merging can take a while when processing a large
executable with a hundred or so hunks.


** ARP **

Programs written for use with the ARP shell are able to specify the
stacksize they require inside the first hunk of their executable. If
such a program is normal or pure imploded, the segment list won't become
visible until the program is run. Thus ARP has no way of finding out what
the proper stacksize should be.
Library imploded programs have no trouble with this because they are
already decrunched after they are loaded into memory with LoadSeg.
(Provided the library has already been made resident.)

The Imploder will recognize these files and report on them. If the
requested stack-size is larger than the usual minimum (4000 bytes)
a warning will be printed, and you'll be urged to use only library
implosion.

The chance of a programmer relying on a soon to be obsolete shell for
setting a stack LARGER than the usual default is rather slim though.
It would have been very nice if 2.0 had sported such a stack setting
feature, and indeed it had been planned, but was never implemented due
to lack of time on the part of the Commodore programmers.

We'll be on the lookout for any future changes to the executable file
format in order to fix any potential incompatibilities before they'll
cause problems.


** The Music **

When we got word the CIA-A timer was used by the OS under 2.0, we switched to
trying to allocate first CIA-A and if not available CIA-B to drive the music.
However the CIA-B interrupt priority is too high and can interfere with things
like serial transfer. So Paul got this great idea to keep on using a CIA for
precision timing purposes, but drop down to a SoftInt for doing the actual
work, modulations, etc. This works great, the amount of code executed under
CIA priority is now negligible.
Recently, the CATS started feeling guilty about hijacking the CIA-A timer and
thus created "Jumpy the magic timer device". If I understood things correctly
the latest 2.0 timer device moves out of the way and starts using a less
accurate timing source whenever an application tries to allocate the CIA-A.
Pauls music driver can run of both CIA-A and CIA-B, and it would be a pity
to make Jumpy jump without good reason, so he changed the alloction sequence
from A-B to B-A.


** The Copperlist **

There are a couple of unavoidable quirks when one uses copperlists on Intuition
screens. On certain machines, probably PAL, under certain circumstances, dragging
a dense copperlist past scanline 256 or so will cause some video crud to appear
at the top of the display. This can't hurt, but it sure does look ugly. I suspect
this is a hardware misfeature because it ain't fixed yet under 2.0
This was the reason why the screen of older Imploder versions wasn't draggable;
you might just think this muck was our doing.

Second problem is that copperlists "shine through" onto other screens in front.
For this reason we've choosen a colour > 4 for the level bars, so this is never
observable with screens less than 3 bitplanes deep.
The 2.0 OS support proper copperlist clipping, but it has been disabled by
default for compatibility reasons (yeach). Supposedly there is a bit somewhere
in the graphics base to turn this back on, so I'm sure, in due time, there will
be some preference editor to re-enable this.


** 68040 Cache Coherency **

With the advent of the 68040 processor, programs that diddle with code which is
subsequently executed will be prone to some problems. I don't mean the usual
self-modifying code causing the code cached in the data cache to no longer
be as the algorithm expects. This is something the Imploder never had a
problem with, indeed the Imploder has always worked fine with anything
upto and including an 68030.

The reason the 68040 is different is that it has a "copyback" mode. In this
mode (which WILL be used by people because it increases speed dramatically)
writes get cached and aren't guaranteed to be written out to main memory
immediately. Thus 4 subsequent byte writes will require only one longword
main memory write access. Now you might have heard that the 68040 does
bus-snooping. The odd thing is that it doesn't snoop the internal cache
buses!

Thus if you stuff some code into memory and try to execute it, chances are
some of it will still be in the data cache. The code cache won't know about
this and won't be notified when it caches from main memory those locations
which do not yet contain code still to be written out from the data caches.
This problem is amplified by the absolutely huge size of the caches.

So programs that move code, like the explosion algorithms, need to do a
cache flush after being done. As of version 4.0, the appended decompression
algorithms as well as the explode.library flush the cache, but only onder OS
2.0. The reason for this is that only OS 2.0 has calls for cache-flushing.

This is yet another reason not to distribute imploded programs; they might
just cross the path of a proud '40 owner still running under 1.3.

It will be interesting to see how many other applications will run into
trouble once the '40 comes into common use among Amiga owners. The problem
explained above is something that could not have been easily anticipated
by developers. It is known that the startup code shipped with certain
compilers does copy bits of code, so it might very well be a large problem.

***************************************************************************

Deplode is a CLI command that allows you to quickly "deplode" imploded
executables.

Usage: Deplode <source> [target]

If no target filename is specified, the sourcefile will be overwritten with
its deploded counterpart.


***************************************************************************


    Disk Imploder V2.27 by A.J.Brouwer.  FreeWare
    ---------------------------------------------


NAME
    DImp - Archives cylinders.


SYNOPSIS
    DImp READ <SourceDrive> Ca[-Cb,Cc,..] <TargetFile> [-M?|-NB|-NC|-X] [+Readme]
      The collection of cylinders to be read must be specified by ranges, Ca-Cb,
      and/or single cylinders, separated by commas.
      -M? or MODE?    = Compression mode, ? = 0-7, default = 5
      -NB or NOBITMAP = Ignore bitmap (normally unused blocks are cleared)
      -NC or NOCOMP   = Do not compress cylinders
      -X  or EXE      = Create self-extracting overlayed executable
      +"Name of Readme file to be displayed during WRITE"
    DImp WRITE <DImpFile> <TargetDrive>
    DImp INFO <DImpFile>
    DImp ABOUT
    DImp will use ".dmp" or ".dex" filename extensions when trying to
    create or recognize DImp data files.


DESCRIPTION
      DImp allows you to compress a subselection of cylinders from
    any floppy compatible device into a file.  DImp is fast and
    sports a good compression ratio.  Notably, DImp allows you to
    create a self extracting file with an appended readme text,
    thus providing you with a completely self-contained method of
    distributing a disk's contents in a file.

       There are four operation mode keywords; "READ" specifies
    that cylinders must be read, "WRITE" indicates that you wish to
    write cylinders encoded in a dimped file, "INFO" displays
    information about the contents of a file containing dimped
    cylinders, and "ABOUT" should be obvious.

      The cylinders are individually compressed, checksummed and
    stored, thus, in case of a transmission error, valid cylinders
    can be undimped, and the originator can then select, dimp and
    retransmit the faulty cylinders.

      The cylinders to be READ can be selected by entering cylinder
    numbers separated by commas, or a dash if you wish to specify
    a range of cylinders.

      When reading or writing, DImp can handle any device that
    supports the normal floppy 2 heads 11 sectors etc. layout,
    thus it is usable with f.e a RAD: or an FMS disk.  You can omit
    the colon normally following these device names.

      Additional options include compression mode selection override.
    Given the fact that the tracks are compressed individually, the
    default mode 5 has a pretty optimal compression ratio, though
    the ratio isn't as good as it might have been if cylinder data
    had been treated as a continuous stream.  This is a design choice
    intended to allow the aforementioned flexible handling of
    individual cylinders.

      If there's sufficient free memory, DImp will allocate buffers
    for a turbo compression algorithm.  Also, reducing the crunch
    mode will speed things up, dramatically so if you don't have
    sufficient free memory for DImp to run in turbo mode.  Lowering
    the crunch modes results in less efficient compression though.

      Next, there are a couple of option switches.  NOBITMAP
    prevents DImp from discarding data that, according to the disk's
    bitmap, isn't used by the filing system.  This option is rarely
    needed.  If you suspect a disk to contain a program that uses a
    low level routine to read its information from that disk, this
    option might be required to prevent loss of relevant data.
    NOCOMP disables compression.  EXE causes DImp to generate self
    extracting dimp files.  These can be executed, and will write
    their contents to a disk selected by the user.  Executable DImp
    files (.dex filename extension) can be treated as normal DImp
    files (.dmp filename extension) in that you can use DImp to
    WRITE these, or get INFO on them.

      Lastly, you may specify a filename containing text to be
    displayed during the writing or self-extraction of the dimp file
    you're creating.  This allows you to include a fairly extensive
    comment on the contents of the disk.  This readme filename must
    follow a "+" commandline indicator.


EXAMPLES
    DImp READ rad 0-79 Work:Cylinders EXE +ram:blahblah


TECHNOCRAP
      The self-extracting executable files generated by DImp are
    overlayed.  This allows for the cylinder data to be spooled
    during the self-extraction process, resulting in much more
    modest memory requirements than the size of the executable
    would lead one to suspect.

      Thanks to resource tracking, transparent error management,
    modular algorithm linking, and sitting on the program for
    several years before releasing it, DImp should be very robust
    and return sensible error messages.

    DImp is pure and can be made resident.


AUTHOR
    Albert-Jan Brouwer
    UUCP hp4nl.nluug.nl!cbmnlux!ecl001!ajbrouw
    UUCP cbmvax.commodore.com!cbmehq!cbmnlux!ecl001!ajbrouw
    FIDO: 2:281/614 (up for freq sometime fall '91)

***************************************************************************


File Imploder V2.33 by A.J. Brouwer
===================================

This program is Freely-Distributable, as opposed to Public Domain.
Permission is given to freely distribute this program provided you
include this documentation and any other related files, and no fee
is charged in excess of reasonable media and mailing costs.


Usage:
FImp <Source> [Target] [-M?|MODE?] [-XO|EXPLODEONLY] [-IO|IMPLODEONLY]
     [-RF|RUNFORMAT "..%s.."] [-AL|ARGLIST "stctsct..."] [-V|VERBOSE]

Modes range from 0 to 11.

FImp is a compress/uncompress command that allows you to reduce
the size of any file. The algorithms used are similar to those in
The Imploder; they sport a good compression ratio and blazing
decompression speed.
You can be easily customize FImp to operate transparently on files
in your environment.


Operation
---------

If you don't specify a target file, FImp will try to replace the
file you specified, and will expect/create imploded files with a
.im extension. Suppose you have a file "test" in the current
directory, "FImp test" will cause it to be replaced with an
imploded file "test.im", and typing "FImp test" once more will
replace "test.im" with the exploded file "test".

Alternatively you can specify a target file or directory.
In the case of a complete target file specification, optionally
with preappended path, FImp will use both the source and target
without any modifications. If you specify only a target directory,
the filename part of the source will be used as a filename.

This works fine because FImp examines the file data to see whether
a file is imploded, and thus can determine if it should proceed with
imploding or exploding.

Note that the ".im" entension is only relevant when you don't
specify a target.  If you do, it will be neither appended while
imploding, nor removed when exploding to a directory.  This means
that it's up to you to specify the ".im" extension for the target
if you prefer to recognize the file type by way of the file name.
Both FInf and FImp have support for recognizing a file as having
been fimped by examining its contents, so if you use batchfiles or
pipe aliases to interface with whatever you've fimped there's no
need to use the ".im" extension. In any case, you can have it both
ways.

The NICE switch causes FImp to operate with a task priority of one
count lower than the one it was invoked with. The old priority will
be restored upon exit. Thus FImp will only eat the spare cycles
without bogging down the system.

If you use the VERBOSE switch, FImp will give a progress report
while compressing, and report on current activities like loading,
exploding etc.


Compression
-----------

FImp doesn't (yet) support streaming compression/decompression; the
file to be compressed needs to be loaded into a memory buffer.
As is the case with The Imploder, a special turbo compression mode
kicks in when you've about 300K memory left (more for higher
compression modes).  This will drastically speed up the implosion
process.

Decompression also requires a contiguous memory buffer as large as
the decompressed file, but doesn't require any memory in addition
to that. Before decompression, FImp performs a quick checksum to
make sure the file hasn't been corrupted.

By default what's most probably the optimal compression mode will
automatically be selected by FImp. If you want to override this, you
may specify the mode yourself using the "mode?" switches. Large
compression modes normally have a higher gain, but take more time,
and, in case of turbo compression, use up a lot of memory. The speed
differences between the various modes are quite large if no turbo
mode could be initiated due to lack of memory.

The large compression modes require quite a bit of memory in order
to be able to execute. If you have some free memory (>300K) but not
a whole lot, and yet want to ensure operation in turbo mode, the
best course of action is to override the automatic compression mode
selection with a relatively small value, e.g. MODE5.


Advanced Features
-----------------

FImp has a number of features that allow you to easily apply it to
customized tasks such as transparently decompressing imploded text
files and loading them into your editor.

The "ImplodeOnly" and "ExplodeOnly" switches allow only compression
or decompression respectively. Normally FImp decides by itself by
looking whether the file is already imploded.

Beyond the "RunFormat" keyword you may specify a command to be
executed when FImp has completed its operation. This isn't very
useful unless you can in some way pass on the files just processed
by FImp.
Therefore this "RunFormat" string allows PrintF like formatting;
Any "%s" format statements present in the string will be replaced
by a filename.

However if you use %s statements you'll also have to specify which
filename it should be replaced with. That's what the ArgList keyword
is all about. Beyond it you may specify a list containing only the
characters "s" and/or "c" or "t".  "c" and "t" are mutually exclusive.
The "s" stands for source, the "t" for target, and "c" for
conditional.
So the ArgList string "st" will replace the first occurance of %s
in the RunFormat string with the source path+filename, and the
second occurance of %s with the target path+filename.

Note that if you specify only a destination directory, a "t" will
still cause replacment by both the path and an appended filename
derived from the source file specification.

Now what's this conditional thing?
Well, if you don't know for sure whether a file has been imploded
and e.g. only care about doing something with the exploded version,
such as loading it into your editor, you can set the "ExplodeOnly"
switch to prevent the file from being imploded when it hasn't
already been imploded.
In this case you'd want FImp to also perform the RunFormat command
(probably containing an invocation string for your editor) even
though it didn't explode a thing. And you'd want the source file
to be loaded into your editor (The one that FImp found out was
already in a non-imploded state).

So this is what the "c" does; It replaces its respective "%s"
in the RunFormat string by the target file name if FImp exploded
or imploded the file. Yet when the file couldn't be processed
due to you setting the ExplodeOnly or ImplodeOnly switches,
the source file name will be used.

One last thing that needs to be mentioned is that the runformat
string supports escaping by means of the \ escape character.
Thus with \" you can include a quote in the runformat string,
and \n inserts a line feed.


Examples
--------

"FImp data"

Will implode the file "data" and replace it by data.im if the
file "data" hasen't been imploded already.
The same command will explode any file "data.im" if present and
replace the exploded version called "data". This way you can
toggle a file between its imploded and exploded state.


"FImp source target -rf "Delete %s" -al s"

This will implode or explode the file "source" to the file "target"
(depending on what state the source file is in) and delete the
source file, but only when the operation could be completed with
success.


The best thing to do is to define some aliases to hide the rather
longish commandlines from view;

Edx=FImp >NIL: [] T:Explode -XO -RF "Execute ExEdit %s" -AL "c"

This will execute a batchfile called ExEdit. This could contain
something like;

<---"S:ExEdit"
.key filepa/a

Edit <filepa>
Delete T:Explode/#?
<---

Thus conditionally loading either the exploded target or the never-
imploded-in-the-first-place source into an editor.
The delete command subsequently purges any intermediate files from
a subdirectory in T:.
Instead of using a batchfile, you may separate commands in the
RunFormat string by using \n line feeds. Still, batchfiles are
more flexible because of e.g. the If/Endif/Skip flow control.


FImp Fish:Contents Fish:Contents -IO

If you don't want to bother with re-imploding files when they
have been modified, you could add a list of these FImp statements
to your CronTab, thus causing re-implosion during nighttime.

As you can see, you may implode or explode a file to itself, but
you need to explicitly specify an identical source and target.
The original file will be deleted, and the new file written. This
results in a "window of vulnerability"; during the write operation
there is no backup of the file.
The replace mode (when you don't specify a target), is safe in this
sense. The source file will only be deleted after the target file has
been successfully written.


General Info
------------

Don't think of FImp as a general purpose archiver. Rather it is
a quick and easy solution to compress single files. I don't endorse
FImped files as a "public" format. There's enough confusion already.

You might use FInf to "add" wildcarding capabilities to FImp,
or to preselect only FImped files from a directory.

The rationale behind all this is that by providing only the building
blocks, documentation and some examples, the user retains the freedom
to implement things as he/she sees fit.


FImp stands out because of its decompression speed. If you'd like
to compress external data used by a program you are writing, while
hardly incurring any performance penalty, consider using FImp.

If you query us, we'll provide you with the decompression routines
in linkable, object format. Note that decompression happens in-situ
and doesn't require any additional memory.

If you wish to make a program that deals with "public" data, e.g.
a text reader or a picture display program, you'd do best to wait
for the xpk library by Urban Mueller. This is an interface library
a la XPR to various compression algorithms. This way people will
have an easy way of decompressing this "public" data without having
to know about the various formats.


FImp is pure and can be made resident.
Enjoy.

# AJ

***************************************************************************

; You can use this alias to read an imploded document.
Alias EdExp FImp >NIL: [] T:Ex -XO -RF "Ed %s\nDelete T:Ex/#?" -AL C

; This does the same thing apart from using FInf to add wildcarding for
; loading multiple files.
Alias Edx FInf >NIL: [] -q -p -f -na -nh -lf "FImp %s T:Ex -XO -AL c -RF \"Ed %%s\\nDelete T:Ex/#?\"" BATCH

***************************************************************************



File INFo manual
================

File INFo V1.131         by Peter Struijk

Introduction
------------

FInf is a very versatile directory listing utility.  It can examine the
contents of files and display a short type description.  In addition to
this, FInf has a whole slew of options that allow you to filter files by
type, date, age, size etc., as well as recursive directory descending,
and adjustable output formatting.  So next to simply listing
directories, FInf is extremely useful for creating hybrid commands that
perform functions closely tuned to your specific needs.



Command Line Arguments
----------------------

Usage: FInf [file/dir] [-lf=LFORMAT ["..%s.."]] [-nl=NOLINE [prefix]]
  [EXE] [IMP] [DOC] [IFF] [NOI] [-t=TYPE string] [-nt=NOTYPE]
  [-na=NOANSI] [-nh=NOHEAD] [NEWER [object]] [OLDER [object]]
  [-d=DIRS] [-f=FILES] [-q=QUICK] [-p=PATH] [-a=ALL] [NOX] [NOC]
  [BLOCK(S)] [DATES] [-lt=LONGTYPE] [SUB string] [BATCH]
  [LARGER size] [SMALLER size] [SINCE [n] date] [UPTO [n] date]
  [-ho=HEADONLY]

Arguments surrounded by double-quotes are NOT recognized as keywords,
this, or the use of -??  shortcuts can help you circumvent problems with
identically named files and keywords.  Keywords/switches are case
independent.

FInf parses its arguments in a left to right order, which may cause
keywords to be overridden by keywords that occur later in the command
line.  E.g.  "FInf -nt doc" displays only text files while "FInf doc
-nt" will disable file type checking.

You may use the backslash "\" as a fixed escape character; If you enter
\n it will be replaced by a new-line, while \" implements a
double-quote.

FInf sets a returncode level of 20 (FAIL) on a serious error.  If no
errors occur, yet no matching file can be found, a returncode 5 (WARN)
will be set.


Examples
--------

In order to whet your appetite, I'll start with a few examples.  A
complete option reference is given later on.

Typing "FInf" without any options will display the files in the current
directory, e.g.

EM                         14228 --p-rwed 12-Jul-90 22:47:39 XLIB IMP
TeX                       130344 ----rwed 04-Feb-90 19:29:42 EXECUTABLE
Introduction                1918 ----rwed 21-Jul-90 15:23:12 ASCII TEXT
Read Me                     1532 ----rwed 22-Jul-90 16:31:52 ASCII TEXT
Graph                       4526 ----rwed 01-Jun-90 18:13:47 IFF PIC
Tmp  (dir)
5 files - 1 directory - 307 blocks - 152548 bytes

Typing "FInf doc" will display all text files in the current directory,
and "FInf disk* exe" will show all executables starting with "disk".

If you use a shell that supports unnamed pipes and aliases, you can very
easily construct new commands as shown below.  If you do not have
unnamed pipes, you can use the BATCH option, the PIPE:  device or
temporary files to obtain equivalent though less elegant functionality.

The BATCH option will cause FInf to output to a temporary file and execute
the contents like a batchfile once it's done.  This will give you the
same functionality as single level unnamed pipes.

------
E=FInf [] noi doc path files lformat " \"%s\"" noline ED | execute
or
E=FInf [] noi doc path files lformat " \"%s\"" noline ED batch

This alias will execute the command ED with a list of text files that
match any wildcards you specify. Typing "E" while in the directory shown
above would have executed the command `Ed "Introduction" "Read Me"'.
Note how escaping the quote character allows you to put quotes around
the produced filename, thus avoiding problems with filenames that have
spaces in them.

Do not be deterred by the numerous parameters. Aliases need to be
constructed only once, and when you are examining the requirements of
a new alias, you'll almost automatically conclude you're in need of
certain options. Chances are FInf provides them.


------

find=FInf NOTYPE FILES QUICK NOHEAD NOANSI PATH ALL SUB []

or abbreviated:

find=FInf -nt -f -q -nh -na -p -a SUB []

This alias will search your current directory, and all subdirectories
for filenames that contain a string you specify.  You can search any
directory by specifying a directory name *after* the search string.
Very useful for finding files on your harddrive.  If a matching file is
found, the full path will be displayed.


------
FInf *.c Quick Path All | Zoo aI <archive>

The I option in Zoo causes it to read filenames from the standard input,
and these are provided by FInf.  This particular example will archive
all C source files in the current directory and its subdirectories.



File Type Filtering Options
---------------------------

FInf has a set of command line switches that causes it to display
only specific file types. Here's a list:


-t=TYPE <string>

When FInf recognizes a file, it is able to print a string describing its
type.  The TYPE option allows you to display only those files where the
pattern <string> occurs in this file type description.  This keyword can
be used to select virtuallly any file type or groups of file types just
by supplying a carefully chosen <string>.  Note that the LONGTYPE option
causes FInf to search the extended file type description instead of
searching the short description.

List of currently recognized file types:

The following list can be generated by "FInf ?" followed by another "?"
at the ":" prompt.  FInf's file type recognition is not by any means
complete and even a bit outdated but serves well to distinguish between
major groups of files like e.g.  executables, text, iff.

SHORT         LONG

UNREADABLE    "Unable to read"
UNKNOWN       "Unknown type"
FIMP DATA     "FImp data file"
OBJ CODE      "Linkable object code"
TTW PIC       "TTW picture"
PW DATA       "PowerWindows data"
DIMP DATA     "DImp data file"
IFF PIC       "IFF picture"
IFF SOUND     "IFF sound"
IFF SCORE     "IFF score"
IFF TEXT      "IFF text"
IFF DMS       "IFF Deluxe Music score"
IFF ANI       "IFF animation"
IFF S3D       "IFF Sculpt 3D scene"
IFF DLV       "IFF Deluxe Video"
IFF DOC       "IFF document"
IFF FILE      "IFF data file"
ASCII TEXT    "ASCII Text"
OLD IMP       "Old imploded"
NORM IMP      "Normal imploded"
PURE IMP      "Pure imploded"
OVLY IMP      "Overlayed imploded"
XLIB IMP      "Library imploded"
XLIB IMP!     "Short library imploded"
EXECUTABLE    "Executable"
OVERLAYED     "Overlayed executable"
NORM IMP*     "Protected normal imploded"
PURE IMP*     "Protected pure imploded"
OVLY IMP*     "Protected overlayed imploded"
XLIB IMP*     "Protected library imploded"
FONT HDR      "Font header"
WB ICON       "WB icon"
DISK ICON     "WB disk icon"
DRAWER        "WB drawer icon"
TOOL          "WB tool icon"
PROJECT       "WB project icon"
GARBAGE       "WB garbage icon"
DEVS ICON     "WB device icon"
KICK ICON     "WB kick icon"


In addition to filtering file types with the TYPE option there are a
number of often used types that may be directly specified by keyword.
These are:

EXE - Display executable files (except imploded executables).
IMP - Display imploded files and imploded data files.
DOC - Display ASCII text files.
IFF - Display all kinds of IFF files. You may also use the TYPE keyword
      to accomplish further differentation.



Other Filtering Options
-----------------------

In addition to type filtering, FInf can also examine other properties
with which to exempt certain files from being displayed:

NEWER <object>
Display only objects which were created after or at the same time as
<object>.

OLDER <object>
Display only objects which were created before or at the same time
as <object>.

UPTO <date>
Display only those objects created UPTO a specified date. UPTO accepts
three formats:
    DD-MMM-YY
    TODAY/YESTERDAY/TOMORROW/MONDAY...SUNDAY/FUTURE
    [n] TICK(s)/SECOND(s)/MINUTE(s)/HOUR(s)/DAY(s)/WEEK(s)/MONTH(s)/YEAR(s)
The integer [n] is optional, default is 1. The latter option is
extremely handy to cleanup your news directory.
E.g. FInf NEWS: -a -p -lf "Delete %s" UPTO 8 WEEKS BATCH
Make sure that your system clock contains the correct time! :-)

SINCE <date>
Display only object since a specified date. For information on the date
formats see UPTO.

LARGER <integer>
Display only files with a size smaller than or equal to <integer> bytes.

SMALLER <integer>
Display only files with a size smaller or equal than <integer> bytes.

SUB <string>
Display only objects containing at least one occurance of <string> in their
objectname.

NOI
Do not display .info files.

-d=DIRS
Display directories only.

-f=FILES
Display only files.

-a=ALL
Recursively displays the contents of any subdirectories in addition to
the files/dirs in the current directory.  Wildcards in the path description
are not supported (yet), so for now this option will cause FInf to
recursively enter ALL encountered directories.



Formatting Options
------------------

These switches and keywords control the way in which FInf generates its
output:


BATCH

This will capture FInf's output into a uniquely named temporary file in
the T: directory, and subsequently will execute the file as a batch file
that will be deleted after completion.
This option is handy if you don't have unnamed pipes and something like
the ARP execute command which will execute any commands piped to it.


BLOCK=BLOCKS

This switch will cause FInf to display the number of blocks files occupy
on disk instead of their lengths.  Note that for devices using the fast
filing system, this value will include any extension blocks.


DATES

This will cause FInf to print absolute dates instead of dates with
"Yesterday" of "Sunday" in them.


-lt=LONGTYPE

If specified, FInf will generate listings with a more verbose type
description.  In order to create printing space for this, the date and
protection flags will not be displayed.


-nt=NOTYPE

If you use this switch, the contents of files will not be examined.  The
type description will therefore not be displayed.  The EXE, IMP, DOC,
IFF etc.  filters won't work if NOTYPE is set.  NOTYPE will cause FInf
to operate about twice as fast because of the decreased amount of work
it has to do for each file.


-na=NOANSI

This surpresses the generation of ANSI codes used by FInf to select bold
or italic font styles in the footer and such.


-nh=NOHEAD

Like with the normal List command, NoHead will stop FInf from generating
headers/footers with additional information on the examined files.

-ho=HEADONLY

This option is included to quickly simulate the unix DU command.
DU stands for Directory Usage, an example alias:
du = finf -ho -a []


NOX

This option removes extensions from the filenames.  This might be handy
when you only wish to pass the filename root into the LFormat output.
E.g.  FInf *.twiddle NOX LFormat "Rename %s.twiddle %s.twaddle" |
execute changes all .twiddle extensions into .twaddle extensions.


NOC

This simple option tells FInf to NOT print any filenotes (comments)
which might be attached to a file or directory.  Note that when the
QUICK option is in effect filenotes are already been ignored.


-q=QUICK

In quick mode, FInf will display only filenames and any paths.  No
lengths, flags, dates, comments or types will be printed, the type
filter switches will still function though.  These bare (path+)file
names can be more readily used by programs you intend to feed FInf's
output to.


-p=PATH

If set, a full path will be pasted in front of any displayed filenames.
This option is useful e.g. in conjunction with the LFormat or ALL
switches.


-lf=LFORMAT <"...%s...%s...">

Works like List's LFormat option.  Briefly, you can specify a string
with a "%s" in it (up to 8 %s expansions are supported), and where FInf
normally would have printed paths+file names, it displays the given
string with the path + file at the %s location.  Useful for pasting
commands in front of filenames so you can pipe the output of FInf to a
batchfile, or directly to execute (if your shell supports this).  E.g.
FInf LFORMAT "rename \"%s\" \"%s.doc\"" FILES DOC BATCH will append a
".doc" extension to all ASCII files found.  Note:  LFormat automatically
causes the Quick, NoAnsi and Nohead switch modes to be selected.


-nl=NOLINE [prefix]

This causes FInf not to print linefeeds between filenames.  If you use
this together with a LFormat " %s" and a QUICK switch, you'll get a line
with file names separated by spaces.  In the prefix position you may
optionally specify a string to be pasted in front of this line.  This
will probably be the name of a command which requires a list of
filenames as its parameters.  Note that this line will only be printed
if any filenames matching your wildcard specification were found.  E.g.
FInf NOLINE "Ed " LFORMAT "\"%s\" " PATH FILES
will produce something like:  Ed "ram:file1" "ram:doc1" "ram:readme"


General Information
-------------------

This program is Freely-Distributable, as opposed to Public Domain.
Permission is given to freely distribute this program provided you
include this documentation and any other related files, and no fee is
charged in excess of reasonable media and mailing costs.

Currently FInf supports the #,?  and * wild cards.  OS 2.0 wildcarding
will be implemented sometime in the future.  Also high on the wish list
is a SORT option switch.  If, during normal operation, there is only one
file matching the file/wildcard specification, and it is a program file,
some additional information about the number of hunks and the memory usage
will be displayed.  FInf has a lot of switches, so there's a chance that
you might want to examine a file or directory that has the same name as
one of these switches.  You can inform FInf that the object you
specified isn't a switch by putting quotes "" around it.

FInf is pure and can be made resident.  Enjoy.


***************************************************************************

; This batchfile demonstrates how to use FInf and FImp in making a "man" online
; documentation batchfile.
; FInf is used for scanning the manual directories. The documentation files
; may be (but need not be) imploded using FImp. Directories are used to
; differentiate between the different reader programs required to display the
; various files.

.key manus/a
.bra {
.ket }

FInf Man:{manus} -q -p -f -nh -na -lf "FImp >NIL: %s T:Ex -XO -AL c -RF \"Ed %%s\\nDelete T:Ex/#?\"" BATCH

FInf Man:ANSI/{manus} -q -p -f -nh -na -lf "FImp >NIL: %s T:Ex -XO -AL c -RF \"More %%s\\nWait 1 sec\\nDelete T:Ex/#?\"" BATCH

FInf Man:IFF/{manus} -q -p -f -nh -na -lf "FImp >NIL: %s T:Ex -XO -AL c -RF \"Run Leggi %%s smart font topaz/8\\nWait 2 secs\\nDelete T:Ex/#?\"" BATCH

***************************************************************************